Add optional video search capabilities to DIVE [DRAFT] - #1743

* Add a Review page that audits annotations as a grid of chips

New top-level Review tab (desktop: after Training; web: after Models)
showing many annotations at once as cropped image chips, so a whole class
(or everything carrying an attribute) can be checked across several
datasets and wrong types corrected in place, without opening each
sequence in the viewer.

- Datasets view / Grid view toggle: pick datasets through the platform
  picker or listing (multicam parents expand into their cameras), then page
  through matching annotations in a rows x columns grid (5x4 by default,
  settable, with zoom in/out keeping the shape) and a context slider for
  the margin around each box (30% by default).
- Query by type above a confidence threshold, or by attribute key/value on
  tracks and/or detections; sort by dataset, confidence or frame. The grid
  keeps its entries until the query is re-run so edits never reshuffle it.
- Chips are cropped client-side from image sequences or by seeking a hidden
  video element (shared frame->time mapping extracted from VideoAnnotator
  into videoSeek.ts), through a concurrency-limited queue that renders the
  first box of every visible entry before any track's extra frames. Track
  entries then cycle through up to 8 boxes sampled along the track, with
  the object kept centred.
- Types are edited per cell (assign, or mark correct) or for a whole page,
  batched and saved through saveDetections; unsaved edits are counted and
  guarded on navigation.
- Clicking a chip opens the viewer on that dataset, seeking to the frame
  and selecting the track (new initialFrame / initialTrackId Viewer props
  read from the route query on both platforms).
- Api gains an optional peekConfig for reading dataset configs without the
  viewer bookkeeping (desktop recents, web browse location); the web
  dataset store now shares its config merge with it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Crop review chips to the grid cell's aspect ratio

A square chip letterboxed into a wide cell wasted most of the cell, so the
crop region now extends the padded square around the box to the cell's
width/height ratio (coarsened to a tenth so window resizes rarely force a
re-render), keeping the object centred whatever the grid shape.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Share grid paging, zoom and controls between chip grids

Move the review page's paging, zoom, chip-resolution and keyboard handling
into useReviewGrid, its grid-shape/context/pager row into
ReviewGridControls, and the persisted grid settings into gridSettings.ts,
so another view showing ReviewItems (e.g. video search results) gets the
same behaviour without copying the page.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Add desktop video search / IQR backend

Per-dataset search index lifecycle (build via a process_video.py job,
status from an index_meta.json sidecar + ITQ file check, delete) and a
persistent QueryServiceManager wrapping viame.core.query_service over
NDJSON stdio (open index, formulate from image chip + boxes, query with
optional warm-start model, refine with +/- feedback, export the SVM as a
runnable trained pipeline in DIVE_Pipelines). One index open at a time;
the embedded postgres is stopped gracefully on app cleanup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Add Video Search side panel with IQR refinement loop

New in-viewer context panel (desktop only) for index-backed video search
and rapid model generation: build/rebuild/delete the dataset's search
index (via the GPU job queue), query from the selected annotation, an
external image file, or a saved .svm model warm-start, adjudicate ranked
results +/- with cropped chip thumbnails and click-to-seek, refine
iteratively, and save good models as runnable trained pipelines. Shared
request/response types live in apispec so web can implement later.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Search across all indexed datasets via one shared index

Replace per-dataset index directories with a single shared search
database (DIVE_SearchIndex): every table keys rows on a per-video
stream identifier, so datasets are added, updated, and removed
independently while one IQR session searches everything at once.

- Add/Update ingests a dataset into the shared database as a job
  (initializing it on first use), re-using stream identifiers derived
  from the dataset id (image sequences) or video filename stem.
- Remove deletes the dataset's rows through the query service and drops
  it from the membership metadata; deleting a dataset from DIVE also
  removes it from the index automatically.
- Query results attribute back to their source dataset via stream_id;
  the panel labels cross-dataset results, seeks within the current
  dataset, crops thumbnails from each result's own media, and can
  filter the display to the current dataset.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Add full-window results grid for rapid search adjudication

New fullscreen 5x4 grid page, opened from the Video Search panel, showing
the ranked results across every indexed video as cropped chips with
accept/reject buttons; marks share the panel's session state and feed the
same IQR refine loop. Chip rendering moves to a shared, concurrency-limited
loader (useResultChips) with context padding and the result box outlined,
and the cell (AdjudicationChip) is presentation-only so future annotation
cluster rows can reuse it.

Also: extracted video frames are now cached per source video (hash-keyed)
instead of colliding across videos on frame number, and cross-dataset chip
media resolves the file the media server actually serves, so transcoded
copies work when originals have moved.

* Cycle track results through sampled frames in the results grid

Track results load their first-state chip immediately, then the grid
queues up to 8 frames evenly sampled along the track (visible page only,
at lower priority than primary chips) and each cell cycles through
whichever frames have loaded, with a filmstrip badge marking animated
cells. Static detection results are unchanged.

* Adapt video search to DIVE main after the rebase

Follow renames that landed on main since the branch was cut: the desktop
project store type is JsonConfig and is read with loadJsonConfig from the
project's datasetFileAbsPath, the frontend metadata loader is loadConfig,
the typed IPC invoke is used in the GPU job queue, and a spawn without a
pid now fails the index build instead of storing an undefined pid.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Show video search results in the shared review grid

The full-window results grid is now the review chip grid: results map to
ReviewItems (searchResultItems.ts), chips are cropped client-side through
the review frame sources (a lazy per-dataset registry, so cross-dataset
results need no backend frame extraction), and the grid shape, zoom,
context and paging come from useReviewGrid / ReviewGridControls with the
same persisted settings as the Review page. Cells keep the accept/reject
actions through ReviewCell's slots and a highlight border; results without
a box show the whole frame.

useResultChips.ts and AdjudicationChip.vue are replaced by
useSearchChips.ts, which also feeds the side panel's row thumbnails.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Build and query file-backed search indexes by default

The shared search index no longer needs an embedded PostgreSQL server:
run_bulk.py --index-backend files writes one set of files per stream
(manifest, descriptor CSV, float32 descriptor array, uids, ITQ hash codes,
tracks) that the query service reads directly. index_meta.json records the
backend the index was built with, every later build joins it, and removing
a dataset deletes its stream files. The PostgreSQL backend is still
supported for indexes recorded (or predating) that backend, through the
same code paths as before.

Also follow the VIAME tool renames (run_bulk.py, database.py,
index_generic.pipe) and drop the initdb requirement from the install check
unless the postgres backend is in use.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Drive search index builds and removals through viame index

The desktop backend now invokes the viame index applet (index.py add /
remove) instead of assembling run_bulk.py arguments and issuing psql
scripts itself: the applet initialises and starts PostgreSQL when that
backend is in use, runs the ingest pipeline, and refreshes the hashes. The
install check looks for index.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review grid: larger cell text, Results first, in-place geometry editing, overlays and sharper chips

The Results panel is now the first and default panel, with the Datasets
panel second; with no dataset added it says so and links across. Cell type
fields and captions scale up as the grid shows fewer entries, and the
threshold and context sliders and fields are larger.

Right clicking a cell (or its edit action) opens the frame it is showing
for editing in place: the box gains drag handles, and polygon vertices and
head/tail points can be dragged too; Enter/Apply keeps the edit, Esc/Cancel
drops it, and the chip is re-cropped around the new box. Polygons and
head/tail points are drawn over every chip. Opening the viewer from a cell
seeks to the frame the cell is showing and selects the track.

Paging quickly only loads the page landed on: queued work for pages passed
over is dropped at once and new loads wait for paging to settle. Chips are
rendered at the cell's resolution with high-quality resampling and stored
losslessly when upscaled, so small objects are as sharp as the source allows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review grid: annotator-style editing handles, right-click confirm, frame stepping and auto-save

Polygon vertices now move while dragged (the draft arrays are replaced
instead of index-assigned, which Vue 2 cannot see). Editing handles are the
circles the annotator draws, in the type's colour and red while dragged, and
a right click while editing locks the change in. A single click on a chip no
longer opens the viewer (double click or the action does), so a slip after
an edit does not raise the unsaved-changes prompt. Track cells gain arrows to
step through their sampled frames, which pauses the cycling until resumed.
When auto-save is enabled in the settings, review edits are saved after the
same delay the annotator uses.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Close the negative-highlight rule left open by the review grid merge

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review grid: settle the cell text between the old and enlarged sizes

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review grid: pause on edit, real-time cycling, overlaid boxes, wheel zoom, live queries and toolbar polish

Starting an edit pauses a track's cycling and it stays paused until resumed.
Cycling runs at the dataset's real-time rate: consecutive frames advance
every 1/fps seconds and sparser samples wait proportionally longer. The box
is drawn over the chip instead of into it, so an edit no longer re-crops the
chip and the view keeps its zoom. While editing, the mouse wheel zooms the
chip about the cursor (up to 16x) and dragging empty space pans it.

The type field's arrow closes its list on a second press, the hover actions
grow under the mouse, entries sort by confidence (highest first) by default,
and the tooltip over each chip is gone. Query changes apply as soon as they
settle, so the Show button is gone; datasets loading or leaving refresh the
grid on their own. A settings gear next to Save opens the annotator's
settings dialog for auto-save. All toolbar fields read at the same size as
the Results and Datasets buttons.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review grid: keep editing handles a constant screen size while zoomed

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review grid: one entry per track across stereo and multi-camera rigs, with add-box and delete actions

The cameras of a multicamera dataset now share a grid entry per track,
shown as one chip per camera side by side and labelled with the camera. The
chips show the same sampled frames on every side; where a camera has no
detection on a frame the track has elsewhere, that side is cropped at a
position interpolated from its own neighbouring boxes, shows no box, and
offers an add-box action that creates the detection there ready to be
adjusted. Type edits apply to the track in every camera and opening the
viewer opens the rig. The chip (image, overlay, editing) is split out of the
cell into ReviewChip so a cell can hold several. A red X next to the accept
action deletes the track, written on the next save.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review grid: cameras of an entry cycle in step, with frame controls kept clear of the camera label

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review grid: keep multi-camera frame controls clear of the camera label

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Add a Query page: index datasets and search them by image, video frame or text

A Query tab (left of Review) with two panels. Datasets lists the datasets
queries search, shows each one's search index status and builds or drops
indexes for several at once, tracking the build jobs. Query searches every
indexed dataset from an exemplar image (with an optional dragged box), from
a frame of a dataset or video file, or from a text prompt swept over sampled
frames with the SAM3 text model, whose hits can seed a similarity search.
Results reuse the video search results grid inline, with refine and saved
models, and open the viewer on double click. The library's selection can be
handed to it.

Two fixes found on the way: a video dataset is now passed to the indexer as
a video rather than an image list (which crashed decoding the mp4 as an
image), and index build jobs are recognised by their real titles, so the
side panel and this page see builds finish.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Add a shared DatasetPicker component

Review, Query, Training, Scoring and Pipelines each grew their own way of
choosing datasets: an autocomplete, a searchable data table with a plus
button, a checkbox table with select all. This adds one component that
covers all of them (search field, list of the datasets on offer with an
add button per row, select all for whatever the search lists, and an
optional browse button for platforms without a listing) so the pages can
adopt it in follow-up changes. Nothing uses it yet.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Use DatasetPicker on the Training and Pipelines pages

Both pages offered their datasets through their own data table with a
plus button (Pipelines also had its own search and select all); they now
share the picker, which keeps the fps column and Training's view button
through the picker's headers and row-actions slot. Training's staged
items are typed as the cache rows they always were.

The picker's search field is clearable, which yields null; filter on an
empty string in that case.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* DatasetPicker: remove all, per-row remove, and hold the list in place

Selected rows now show a clickable check that removes them, and a
"Remove all (n)" button drops every listed selection, mirroring select
all. Adding or removing keeps the picker at the same place on screen:
the pages list the selection above it, so growing that list used to
push the picker down under the user's pointer.

Training and Pipelines section titles and descriptions lose their card
padding so they line up with the tables and the picker under them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Pipelines: list datasets on entry and put the selection under the picker

The available datasets no longer wait for a pipeline to be chosen; the
run button stays disabled until one is. The selected datasets and the
run button move below the picker, so adding to the selection no longer
pushes the list the user is working in down the page. That makes the
picker's scroll hold unnecessary, so it is removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Pipelines: list every dataset before a pipeline is chosen

getAvailableItems still returned nothing without a selected pipeline,
which left the picker empty on entry despite the section now showing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Training: put the selected datasets under the available list

Same order as the Pipelines page: configuration first (name, config
file, labels, the annotated-frames and fine-tuning options), then the
picker, then the selected datasets with the Train button. Interrupted
runs follow the selection instead of splitting the configuration from
the datasets.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review: choose datasets with the shared DatasetPicker

The Datasets view drops its autocomplete for the picker used by Training
and Pipelines: search, add or remove per row, select all and remove
all, with the web's folder browser behind a Browse button. The selected
datasets keep their status table underneath.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Query: choose datasets with the shared DatasetPicker

The Datasets view drops its autocomplete for the picker used by Review,
Training and Pipelines. The selected datasets keep their index status
table with the build controls underneath.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review: queue datasets picked on the Datasets view until Results opens

Adding a dataset used to read its annotations at once, so picking many
on the Datasets view meant waiting for every one to load before doing
anything. Picked datasets are now queued and load together when the
Results view opens (or on reload). Datasets handed over by the library
still load immediately, as results are shown right away.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Query: build indexes for the selected datasets without a second selection

The Datasets view asked for datasets twice: pick them for the list, then
tick them again to build indexes. The checkboxes are gone; Build index
covers every selected dataset not indexed yet, and each row has its own
build button.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review: wheel zoom and drag pan at any time, shared across an entry's cameras

Zooming a chip no longer requires edit mode: the wheel zooms about the
cursor and dragging the zoomed image pans it, with the action buttons
left alone. Multi-camera entries hold one view in the cell, so zooming
or panning one camera moves the others with it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review: offer only types present at the current threshold

The type dropdown listed every type on any confidence pair of the
selected datasets, including low-confidence secondary classes that a
query at the current threshold would never match. It now lists the
types some track carries at that confidence or above, so every choice
has results; the cell type field keeps offering every known type.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review: own type dropdown per entry, and middle-button panning

The entry type field used a native datalist, whose arrow could open the
list but never close it. A small dropdown of our own replaces it: the
arrow toggles, typing filters, arrow keys and Enter pick, Escape
reverts, and the list floats out of the clipped cell.

The middle mouse button now pans a zoomed chip from anywhere on it, in
or out of edit mode, with the browser's autoscroll suppressed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Review: resume the session when coming back to the page

Leaving the Review page hands its service, view and grid page to a
held session that the next visit takes back, so navigating away and
returning shows the same datasets and page, unsaved edits included.
Starting Review from the library with a different selection begins a
fresh session, unless the held one has unsaved edits, in which case it
is resumed and the new datasets are added. The route-leave prompt goes
away since nothing is lost; closing the app still warns.

The entry type field also commits a typed type on Enter directly
instead of relying on the blur that follows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

* Track indexing preparation and logs in Jobs and index first stereo camera

* Reliably show indexing failure dialogs including preparation errors

* Centralize index creation on Query and select sequence search indexes

* Retain index rows and job outcomes across Query navigation

* Separate successful index entries from pending builds and expose deletion

* Launch sidebar video searches on the Query page

* Query page: whole-image searches, Index panel cleanup, whole-frame indexes

Run the query when formulation returns no results (whole-image exemplars).
Hide indexed datasets from the picker, show each entry's index type, add
the whole-frames index method, and rename the Indexes toggle to Index.

* Left-align the Index panel tables with their section titles

* Edit search results as annotations and hide reviewed ones

Typing a type or editing a box adopts a result into its dataset through the
review service, linking to an overlapping annotation or inserting a new track.
Save/Discard on the results toolbar; Hide reviewed drops adjudicated entries.

* Keep the accept and reject actions green and red on search results

* Resume the Query page where it was left

Park the search session, review edits, chips, view and grid position when
leaving the page and take them back on return, so adjudication continues
after opening a result in the viewer.

* Save search results by adjudication and resolve overlaps with originals

Accepted results and typed ones are written (rejected only when typed); each
gets its own track instead of editing existing annotations in place. Results
overlapping originals prompt: keep originals, replace overlapping, replace all, or discard.

* Lay the overlap dialog out as a 2x2 grid of captioned choices

* Wrap the Video Search launch buttons and move Save model beside Save Annotations

* Stack the Query page toggles in the controls column so results start at the top

* Give text query hits the same results card and toolbar as image and video searches

* fix unmounted effects not saving changes

* modify for actual prompt

* linting

* Wire onlySelected filtering into the Query results grid.

Pass page.results into createSearchChips so the toggle actually limits which hits appear for adjudication.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Remove multicamera child streams when deleting a parent from the search index.

Match parent/camera dataset ids as well as exact ids so index cleanup does not leave orphaned streams.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Add /api/video-info so Query can probe external video files.

Frame extraction needs fps from disk; wire ffprobe through the desktop backend route the frontend already calls.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Clear the video exemplar box when the frame or source changes.

Avoid searching with a box drawn on one frame against an image from another.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Allow saving accepted whole-frame search results without boxes.

Adopt boxless hits as frameless keyframes so marking them correct can persist on save.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Plan text-query frames from probed video length.

Use video-info duration and dataset fps so text search stays within the clip instead of aborting past EOF.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Retry ensureLoaded after a dataset load error.

Failed loads no longer stick in error until the Query session is restarted.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Clear stale Query results after index jobs and scope search saves.

Index builds close the backend session, so invalidate the in-memory
query session (results, adjudications, generation) when jobs finish or
membership changes. Pass the filtered results list into search review
so change counts and saves only touch hits the grid is showing.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Treat EPERM/EACCES as a live process when probing training PIDs.

process.kill(pid, 0) can fail with permission errors for a still-running
job (other uid or a restricted environment). Only ESRCH means the
process is gone, so resumable-training discovery no longer misclassifies
active runs as interrupted.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix typecheck

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Bryon Lewis <Bryon.Lewis@kitware.com>
Co-authored-by: Cursor <cursoragent@cursor.com>